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Abstract 

Response to Intervention (RTI) is an educational approach that integrates 
ongoing assessment of individual student progress with targeted instruction. 
Administrators and teachers in P-12 schools expressed a need for colleagues 
in higher education to provide training to general education pre-service and 
in-service teachers in selecting appropriate instruments and conducting 
accurate assessments, tasks traditionally performed by special education 
teachers. Researchers developed a scale to measure self-efficacy of educators 
using these approaches to better identify areas in which educators need 
additional support. Researchers wanted to know if the scale functioned as 
expected and if it was appropriate to use the scale for their intended purposes. 
This paper describes the results of the study of the characteristics of the scale 
following the administration of the pilot, including indices of score reliability 
and utility. The researchers used measures of internal consistency and factor 
analysis to assess scale quality. The results indicate that the scale is useful 
for measuring teacher perceptions of their self-efficacy using multi-tiered 

instructional approaches. 
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hen asked to describe the education system in the United States, one of the first issues 
that respondents address is that there is not one, unified system. A typical description would 
start with clarifying that there are separate systems for P-12 schools and postsecondarv 
education. 


Some states are working toward integrating these systems, with the goals to reduce 
resulting disjuncture and improve instruction (Minnesota, 2002). At the heart of improved 
instruction, at any level, is teacher quality. A key link between the systems of higher edu¬ 
cation and P-12 is found in teacher education programs, particularly in the area of assess¬ 
ment. Assessment issues in P-12 and post-secondary systems are similar. Practitioners 
in both systems collect and use performance data to inform accountability systems and 
to improve instruction. Efficiently planning to meet the instructional needs of in-service 
teachers, known as professional development, of pre-service teachers in teacher prepara¬ 
tion programs, and of P-12 students requires the same thoughtful processes - assessing the 
learners’ needs, planning and implementing the appropriate intervention, evaluating the 
effectiveness of that intervention, and making revisions in subsequent instruction based on 
the outcomes. In public schools, that process is known as Response to Intervention (RTI). 

RTI is a 2004 federal public education regulation requiring educational practices 
designed to narrow achievement gaps and meet the needs of all students (Individuals with 
Disabilities Education Act of 2004). Data concerning a child’s response to instruction and 
interventions can be used to guide instructional and behavioral decisions and even eligi¬ 
bility for special education services. Implementation of RTI practices requires more than 
“tweaking existing assessment practices” but instead necessitates systems change (Burns 
& Ysseldyke, 2005). 
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While the 2004 regulations do not mandate a multi-tiered instructional model, 
RTI practices do not work without implementation within a multi-tiered instructional 
model (MTI). In an MTI model, educators design instruction with well-integrated content, 
goals, evidence-based instructional practices and assessment practices for best benefit to 
most learners in the general education setting. When students struggle with core instruc¬ 
tion, educators reteach content to appeal to varied learning styles or to fill learning gaps. 
When students do not respond to reteaching, educators intervene with tiered interven¬ 
tions at varied levels of intensity, first providing strategic interventions in small groups 
and when necessary providing intense interventions in very small groups or individual¬ 
ized to meet the unique needs of learners. MTI is a system involving collaborative partner¬ 
ships between classroom teachers, specialists and administrators. 

Among other practices, MTI stresses evidence-based practices and data-driven 
decision-making (Barnes & Harlacher, 2008). Some practices are not so new, such as 
collaboration, though MTI pushes collaboration to new levels. With an emphasis in early 
interventions to address struggles before gaps reach serious levels, specialists may play 
proactive roles in core instruction, interventions, or assessment structures. Other MTI 
principles may feel new to some teachers such as data-driven decision-making, and imple¬ 
menting tiered interventions to meet individual needs. 

Public schools requested support in providing professional development for RTI 
and MTI practices. Those requests were non-specific. In order to design appropriate pro¬ 
fessional development, the researchers started with a needs assessment. They reviewed 
literature to determine core content knowledge and skills used in RTI and MTI approaches 
and to find assessment instruments to measure needs for training in those areas. Various 
checklists exist to evaluate school or district-level implementation of a multi-tiered in¬ 
tervention methods or RTI practices such as Florida’s Self-assessment of Problem-solving 
Implementation (SAPSI) or Kansas’ Innovation Configuration Matrix (Florida Problem- 
Solving/Response to Intervention Project, 2008; Kansas State Department of Education, 
2009). Each of these checklists works as a tool for schools or districts to evaluate system¬ 
atic levels of progress toward or implementation of various practices such as assessment 
practices. At the beginning of this project, a review of available scales determined that no 
one scale or combination of scales effectively assessed self-efficacy in the unique compo¬ 
nents of MTI practices. 

During the time of this study, Florida published the Perception of RTI Skills Sur¬ 
vey, a self-rating scale used by teachers to evaluate skills specific to RTI practices such 
as hypothesizing reasons for gaps and determining appropriate interventions (Florida 
Problem-Solving/Response to Intervention Project, 2008). Nunn and Jantz (2009) re¬ 
cently demonstrated that the Teacher Efficacy Beliefs and Behavior Scale (TEBBS; 1998) 
scores have validity for measuring general teacher self-efficacy. Nunn, Jantz and Butikofer 
(2009) further demonstrated that the TEBBS positively correlated with one measure of 
student outcomes, the Indicators of RTI Effectiveness Scale (Nunn, 1999). 

While the impetus of this study was to assess professional development needs for 
in-service educators, this study provided valuable applications for teacher education pro¬ 
grams. Pre-service educators must be prepared to enter their profession fully equipped to 
meet the varied demands of MTI practices. Therefore, the research focus on professional 
development needs of in-service teachers provided important insight for teacher educa¬ 
tion program development. 

The instrument developed and piloted through this study, the Multi-tiered In¬ 
struction Self-Efficacy Scale (MTISES), specifically assesses teacher self-efficacy for MTI 
practices using a survey taking approximately ten minutes to complete. Because the first 
version, the Response to Intervention Self-efficacy Scale (RTISES), was a new instrument, 
the researchers wanted to know if the scale functioned as expected and if it was appropri¬ 
ate to use the scale for their intended purposes. 


“At the heart of improved instruction, at any level, is teacher quality.” 
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Research Questions 

1) Does the scale measure one broad construct or several more specific constructs 

that can be used to characterize self-efficacy using MTI approaches? 

2) What are the meanings of the factors that account for the variation among the set of 

items? 

3) How can the scale be used for planning professional development in using MTI 

approaches? 

This report addresses these questions using several methods, including descriptive and 
factor analyses. 


Participants 

Participants in the scale development process included educators from two school 
districts, teacher education faculty, and university psychometric experts. Psychometric 
experts included doctoral students in an assessment and measurement program and one 
university professor in educational psychology assessment. The teacher educators came 
from departments of special education and of general early and elementary education. 
Both school districts are rural with farming communities and small towns. One of those 
districts served as an MTI pilot district, fully engaged in implementation of MTI practices. 
The second school district was in early stages of RTI planning. Participants in the pilot of 
the instrument included educators from three school districts, two fully engaged in MTI 
implementation. The 184 survey respondents included teachers, specialists, and adminis¬ 
trators. 


Instrument Development 

In order to develop a scale with practical and accurate value for educators and 
professional development trainers, researchers followed the DeVillis scale development 
process (2003). That process follows eight steps: (1) decide what to measure, (2) gener¬ 
ate item pool, (3) format the measurement, (4) have item pool reviewed by experts, (5) 
consider validation items, (6) administer items to a developmental sample, (7) evaluate 
items and scale quality, and (8) determine optimal scale length. 


Determining Constructs and Items 


“RTI is a 2004 federal 
public education 
regulation requiring 
educational practices 
designed to narrow 
achievement gaps and 
meet the needs of 
all students.” 


The growing body of literature on RTI and MTI-related issues guided the content 
for the first two steps, focusing on five core constructs. These constructs represent emerg¬ 
ing MTI practices, the areas in which teachers would most likely need to revise familiar 
methods used for assessment and instruction within their classrooms. The researchers 
identified those five constructs as universal design for learning, proficiency in judging ev¬ 
idence-based practices, collaboration, data-driven decision-making, and implementation 
of interventions. Universal design for learning (UDL) emphasizes proactive instructional 
design to address needs of all learners in varied presentation of material, multiple ways to 
engage with learning, and multiple expressions of learning. UDL respects varied learning 
styles, ability levels and/or language competencies (Strangeman, Hitchcock, Hall, 8c Meo, 

2006) . Proficiency in judging evidence-based practices includes the need to find what 
practices are research-based, to judge appropriateness for populations and purposes, and 
to evaluate effectiveness based upon the research (Barnes 8c Ilarlacher, 2008). MTI may 
change the degree of collaboration (Burnes & Coolong-Chaffin, 2006; Leaving No Child 
Behind, 2007). Data-driven decision-making requires educators to find or create appropri¬ 
ate assessment tools, gather meaningful assessment data, and interpret and make deci¬ 
sions based upon data (Barnes 8c Ilarlacher, 2008; Frey 8c Fisher, 2004; Fuchs 8c Fuchs, 

2007) . Finally, educators must implement small group or individualized interventions in 
tiers of increasing intensity to meet the specific needs of individual learners (Fuchs 8c De- 
schler, 2007; Mellard, 2008). Though MTI incorporates many practices of good teaching, 
these five components emerge as areas requiring refinement of practice. 
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Scale refinement. To help maximize item appropriateness, scale developers had 
all items reviewed by experts for relevance to the area of interest, MTI practices. Three 
focus groups participated in this part of the scale development process. The first focus 
group consisted of two university faculty who had researched MTI practices, and one 
experienced teacher. A second focus group consisted of general and special educators, 
specialists, and administrators active in MTI leadership. Focus group participants were 
asked if all relevant issues related to self-efficacy using MTI practices were represented 
and if there were items that needed to be added or omitted. This item review process was 
one way the researchers addressed the concern of sampling the content of this new area 
and confirming their theoretical framework of self-efficacy using MTI practices that they 
had constructed based upon their review of literature and professional experiences. Par¬ 
ticipants shared feedback about specific items, the scale as a whole, and the time required 
to complete the questionnaire. 

The third focus group consisted of psychometric experts, two doctoral students in 
psychology assessment and their professor. In multiple sessions, that focus group mapped 
items to constructs, evaluated wording of items and response options, critiqued valida¬ 
tion items, and required defense of items, allowing for elimination or refinement of items. 
During this scale refinement phase, one debated issue was the labeling of the anchors on 
the response scale. Several configurations were discussed including a sliding scale upon 
which respondents could place a marker indicating their level of agreement to statements 
regarding perceived competency in a particular area. Other options included language 
such as, “I do not know how to do this” and “I am an expert at this” to indicate levels of 
self-efficacy. Each proposed scale generated concerns from either the psychometricians 
or the teachers. The goal was to use language that would he understood and used consis¬ 
tently among the educators so that the results could be interpreted meaningfully. Interest¬ 
ingly, the focus group participants in this process helped to create a response option very 
similar to Florida’s Perception of RTI Skills Survey (Florida Problem-Solving/Response to 
Intervention Project, 2008) though that study was published after this stage of this study. 
Focus group participants in this study justified answers ranging from “I’ll take anything” 
to “I’m ready to help others,” motivated by a desire to offer options which would limit 
defensiveness yet focus on self-efficacy for the specific behaviors. The initial version, the 
RTISES, is found in Appendix A. 

Scale piloting. Finally, the RTISES was piloted using web-based survey software. 
Participants included three university faculty and 184 educators in three school districts. 
Most respondents served students in kindergarten through second grades (n=79, 42.2%) 
and/or third through fifth grades (n=71, 38%) with 31 respondents serving all grades 
(16.6%) and only three serving middle school or secondary grades (.5%). Survey partici¬ 
pants included 87 general educators (46.5%), 38 special educators (20.3%), with 43 (23%) 
serving all students, and the rest serving specialized target populations. 

Scale Quality 


Reliability of Scale Scores 

Procedure. The reliability of the scores from this new instrument was examined. 
First, to check the homogeneity of the items, a test of internal consistency was performed. 
The goal was to achieve a Cronbach’s alpha of at least .90. Next, the item-total correlations 
were calculated. The goal was to have Pearsonian item-total correlations over .3. 

Residts. Cronbach’s alpha based on the 58 standardized items was .976. Appen¬ 
dix C provides the results of the Pearsonian item-total correlation. Of these 58 items, 57 
of them had correlation coefficients of over .3, most between .6 and .8. All correlations 
were statistically significant at the .001 level. 
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Constructs Characterizing the Item Set 

Procedure. The researchers conducted a preliminary check to see if the set of 
items measured one broad construct, self-efficacy using MTI approaches, or several more 
specific constructs, such as the five areas explored in the item generation process. First, 
an 8-item subscale measuring the construct of general self-efficacy was included in the 
pilot to provide additional understanding of how the new items related to this general 
measure (Schwarzer & Jerusalem, 1993). The researchers expected that the responses 
to this subscale would be related positively to the responses on the new RTISES. Next, 
researchers examined the results of the factor analysis that used principal component 
analysis to generate initial values. 


Residts. The correlation between general self-efficacy subscale score and the 
RTISES total score was positive, but not strong: r( 155) = .14, p =.08. While the correlation 
was not significant relative to the standard alpha level of .05, the p-value was less than .10. 
These eight items were not included in further scale analysis. 


“Implementation of 
RTI practices requires 
more than ‘tweaking 
existing assessment 
practices’ but instead 
necessitates systems 
change.” 


The factor analysis extracted 10 factors with initial Eigenvalues greater than one, 
however, there is one predominate component, initially labeled by the authors as self- 
efficacy in using MTI approaches, explaining nearly half of the variance. This result pro¬ 
vides encouragement for future work continuing to gather evidence to support a claim of 
unidimensionality of the construct (see Table 1, Figure 1 and Appendix B). Recall that the 
scale was designed to address the a priori framework of five components comprising self- 
efficacy using MTI approaches and that each item essentially appears twice - as an item 
addressing the need for information and as an item addressing the need for training in the 
instructional method. Therefore, the criteria number for factor extraction was set to five, 
instead of ten. The Rotated Component Matrix (Appendix D) shows how items loaded on 
five factors. SPSS output generated the labels Components 1 - 5 on the matrix. Bold type 
has been used in the matrix to flag strong values and to facilitate defining the substantive 
meaning of the factors that account for the set of items. Titles were assigned to the groups 
of items and used in Figures 2 and 3. 


Table 1 

First 10 Eigenvalues 


Component 

Total 

% ofVariance 

Cumulative % 

1 

24.947 

43.013 

43.013 

2 

4.808 

8.289 

51.302 

3 

4.187 

7.220 

58.521 

4 

3.581 

6.174 

64.696 

5 

2.420 

4.173 

68.869 

6 

2.226 

3.838 

72.707 

7 

1.531 

2.640 

75.347 

8 

1.295 

2.233 

77.580 

9 

1.192 

2.055 

79.635 

10 

1.070 

1.845 

81.480 


Examination of the Component Matrix revealed that the items did not load as 
expected based on the theoretical design of the issues related to self-efficacy in using the 
MTI approaches. Recall that the instrument was designed with five components in mind 
- universal design for learning, proficiency in judging evidence-based practices, collabora¬ 
tion, data-driven decision-making, and implementation of interventions. All items loaded 
positively on the first factor. Items addressing how to adapt learning activities to engage 
English Language Learners (ELLs) and how to allow ELL students to demonstrate learning 
loaded on a factor that had not been anticipated in the theoretical framework. Collabora¬ 
tion with grade level team members, items!5 and 16, loaded on two different factors. 
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Scree Plot 



Component Number 

Figure 1. Components extracted by SPSS factor analysis from initial scale. 


These items loaded with items addressing collaboration with professionals outside of the 
grade level teams and with using universal design. This analysis provided some evidence 
that self-efficacy in using the MTI approaches is not one broad construct, but rather sev¬ 
eral more specific ones. Using the information from loading patterns, the authors labeled 
the factors with titles descriptive of the items found there - universal design to teach and 
engage learners, meeting the needs of English language learners, seeking evidence-based 
support, collaboration, and using data for progress monitoring and implementing solu¬ 
tions for students. Figures 2 and 3 illustrate the a priori and new frameworks. 



Figure 2. Theoretical model of structure of the RTISES. 
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“Proficiency in judg¬ 
ing evidence-based 
practices includes 
the need to find what 
practices are 
research-based, 
to judge appropriate¬ 
ness for populations 
and purposes, and 
to evaluate effective¬ 
ness based upon 
the research.” 



Figure 3. Pattern of items loading on five components of the RTISES. 


The authors examined the correlation matrix for additional information to ad¬ 
dress scale quality. The extremely high correlations between the items looking at educa¬ 
tors’ perceived needs for more information and their perceived needs for related training 
(see Appendix E) prompted a closer exploration into scale length and the possibility of 
removing items without losing important information. The instrument design presented 
questions as sets of paired items wherein respondents were first asked to address their 
need for information in a particular area and then asked to address their need for training 
in that same area. Careful review of the correlations between the two items revealed that 
the bifurcated questions addressing information and training could be collapsed into a 
single item, thus reducing the scale by half. Because the purpose of the scale was to inform 
professional development needs, the items addressing information were eliminated and 
further analysis used the data from the items measuring the need for training. 

One item addressing behavior did not fit with other items. While the other items 
did not specifically address teaching and learning in a strictly academic or cognitive pro¬ 
cesses domain, the implication was there. The stand-alone item (Q24) that addressed 
behavior in the social-emotional domain was dropped from the scale. 

The authors analyzed how well the training items function without their compan¬ 
ion information items. Reducing the number of items would benefit the survey respon¬ 
dents by reducing time needed to respond to the questions, but longer scales typically 
have higher reliability estimates. To estimate reliability, researchers calculated Cron- 
bach’s alpha for subscales to measure internal consistency and to evaluate how well these 
new subscales functioned. Cronbach’s alpha reliability coefficients range between 0 and 
1, with higher values indicating greater internal consistency. The results for this study are 
found in Table 2. Using the guidelines provided by a SPSS handbook (George & Mallery, 
2005) the alpha values for these five new subscales (minimum alpha = 0.789 and maxi¬ 
mum alpha = 0.925) are considered to be very good to excellent. The measure of reliabil¬ 
ity for the total scale, the Cronbach’s alpha for 28 items, is .952, a very strong indication 
of overall internal consistency, but not an absolute indication of unidimensionality. 

Factor analysis using just the training items extracted six factors with initial Ei¬ 
genvalues greater than 1, and one predominate component explaining 45 percent of the 
variance (See Figure 4). The Rotated Component Matrix (Appendix F) shows how items 
loaded on six factors. Bold type has been used in the matrix to flag strong values and to 
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Table 2 


Item-Total Correlations for Subscales 


Subscale 

Number 
of Items 

Items 

Cronbach’s 

Alpha 

Differentiation to Assess and Engage Learners 

6 

1,2, 4, 5,7, 8 

.914 

Meeting Needs of English Language Learners 

3 

3,6,9 

.789 

Seeking Evidence-based Support 

5 

10, 11,12, 13, 14 

.925 

Collaboration 

4 

15, 16, 17, 18 

.861 

Data-driven Decision Making 

10 

19, 20,21,22, 23, 
25,26, 27,28,29 

.911 

Total Scale 

28 


.952 


User-defined missing values are treated as missing. 

Statistics are based on all cases with valid data for all variables in the procedure. 


facilitate defining the substantive meaning of the factors that account for this smaller set 
of items. Examination of the Component Matrix revealed that using this reduced scale, 
the items loaded nearly as expected based on the theoretical design of the issues related 
to self-efficacy in using the MTI approaches. The five initial components (universal design 
for learning, proficiency in judging evidence-based practices, collaboration, data-driven 
decision-making, and implementation of interventions) and the additional component 
addressing how to engage and assess English Language Learners are represented here. 
Collaboration with professionals outside of the grade level teams emerged as a separate 
component. Titles were assigned to the groups of items and used in Figure 5. 


Scree Plot 



Component Number 


Figure 4 . Components extracted by SPSS factor analysis from revised scale. 
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“Respondents seemed 
to feel that meeting 
the needs of English 
language learners is 
different from meeting 
the needs of other 
learners and that 
behavior is a different 
concern than academic 
purposes.” 
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Figure 5. Pattern of items loading on six components of revised MTISES. 


Utility. In this small scale study, the researchers demonstrated the value of the 
RTISES as a measure of teacher self-efhcacy specific to MTI practices, especially for the 
purpose of professional development needs analysis. The resulting MTISES worked to 
measure teacher self-efficacy for MTI practices in five specific areas of MTI practices. 
One next step is to determine the utility of this scale for similar purposes in a larger scale 
study. A second future step is to determine the utility of subscales of the MTISES for 
pre-post measures of gains made in response to professional development in those areas. 
Professional development modules and courses are currently under development for com¬ 
ponents measured by the MTISES. Therefore, the researchers intend to study the utility 
of pre-post assessments connected to professional development modules and courses on 
each separate subscale of the MTISES. 

Discussion and Practical Applications 
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“Improving teacher 
quality through better 
teacher preparation 
and development 
is one of many 
ways that the P-12 
and postsecondary 
education systems 
can collaborate.” 




•RPA Volume Six I Winter 

M 


2011 


The initial results of the instrument quality review provided some evidence that 
the resulting MTISES (Appendix G) measures teacher self-efficacy in using MTI approach¬ 
es. Careful scale construction processes were used to maximize item appropriateness. 
The direction of the relationship between this scale and another scale of general self- 
efficacy were the same, but not highly correlated, indicating self-efficacy in these practic¬ 
es is different from general self-efficacy. Internal consistency was strong for the subscales 
and for the overall scale. It appeared that the areas in which the teachers saw the need 
for professional development did not align precisely with the conceptual map envisioned 
by the investigators. Specifically, respondents seemed to feel that meeting the needs of 
English Language Learners is different from meeting the needs of other learners and that 
behavior is a different concern than academic purposes. Using data from this develop¬ 
mental sample to investigate optimal scale length, the researchers found evidence that the 
scale works well with half of the RTISES items removed. The subscale responses indicated 
areas in which teachers felt they needed additional professional development. 

The MTISES has practical applications for teacher education programs gathering 
data for accreditation purposes. Such accreditation is earned through meeting require¬ 
ments of such organizations as the National Council for Accreditation of Teacher Educa¬ 
tion (NCATE) or the Teacher Education Accreditation Council (TEAC). Each accrediting 
agency requires data demonstrating that pre-service teachers have obtained quality 
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levels of knowledge and skills relevant to teaching practices and evidence of value added 
through program participation (National Council for Accreditation of Teacher Education, 
2011; Teacher Education Accreditation Council, 2011). One northeastern college’s teach¬ 
er education program is currently using the MTISES to assess student gains in compo¬ 
nents of multi-tiered instructional practices through participation in a course and paired 
field-based experience. 

Professional development on specific components of RTI and MTI is essential 
to successful implementation of RTI and MTI practices. Various experts have proposed 
models for such professional development (Brown-Chidsey & Steege, 2005; Kratochwill, 
Clements & Kalymon, 2007; Kratochwill, Volpiansky, Clements, & Ball, 2007). To respond 
to the practical needs, the researchers are collaborating with experts in higher education 
institutions to post on-line professional development modules offering continuing educa¬ 
tion credits for educators and meeting introductory-level needs of both pre-service and 
in-service educators. These settings will provide opportunities for researchers to gather 
objective measures of the teachers’ competencies implementing MTI approaches. For 
example, teachers could demonstrate their ability to interpret student assessment data 
before and after receiving data analysis training and then their scores on these assess¬ 
ments could be compared to the self-reported, self-perceived ability to do the same task. 
Objective measures will give researchers insight into the relationship between actual and 
perceived skill levels. 

Ongoing follow-up studies using the instrument to measure a change in the level 
of self-efficacy before and after professional development will add to the fuller understand¬ 
ing of the utility of the scale to measure change and the effectiveness of training. One large 
mid-Atlantic urban school district proposed use of the MTISES as one pre/post-assess- 
ment of effectiveness of new professional development initiatives. One research study in 
a southern state is currently using the MTISES as a pre/post- assessment instrument con¬ 
nected with district-wide professional development. 

As both in-service training programs and pre-service teacher education programs 
implement professional development for MTI practices, instruments such as the MTISES 
are essential for identifying training needs and measuring gains in response to profes¬ 
sional development. Results from all of these studies should demonstrate the utility of the 
MTISES for use in measuring change over time in response to professional development 
through post-secondary education. 

Improving teacher quality through better teacher preparation and development 
is one of many ways that the P-12 and post-secondary education systems can collaborate. 
Summaries by the Education Commission of the States (ECS) describe other collaborative 
efforts, referred to as K-16 initiatives, including programs that focus on aligning standards 
and policies that develop cross-system structures (ECS Education Policy, 2011). Higher 
education assessment professionals are key stakeholders in the area of improving student 
learning by leading efforts to educate teachers, and also by making sure that their teacher 
preparation programs align with the P-12 curriculum and with licensure processes. 

Conclusion 

This study addressed key characteristics of the MTISES, specifically score reli¬ 
ability, the question of whether multi-tiered intervention self-efficacy has more than one 
specific construct, and the potential utility of the instrument. This initial administration 
and preliminary analysis of the MTISES provides researchers with guidance for further 
study, especially in the area of measuring change in self-efficacy after training. This work, 
along with repeated administrations of the test to increase the sample size, will add to the 
increasing evidence of construct and content validity of the scores. 
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Appendix A 

Response to Intervention Self-Efficacy Scale 
The original version, RTISES 

All scale items use the following response option: 



I'll take 
anything 

I'm starting to 
get it, but 1 
want lots 

more 

1 do this, butl 
could benefit 
from more 

1 don't feel the 
need for more 

Ifeel ready to 
help others 

information 

O 

O 

O 

O 

O 

training 

O 

O 

O 

O 

O 


DIRECTIONS: 

For most of the following questions, you will be asked to indicate your needs for information and/or training in various 

educational practices. For each question, please indicate first how much more information you desire on that topic, and 

then how much more training you desire on that topic. 

For purposes of this survey, information means resources you can process on your own through print or web-based 
resources. 

For purposes of this survey, training includes such supports as mentorship, coaching, workshops, conferences and 
courses. 

1. Flow much information and/or training do you need about differentiating presentation of information for various 
learning styles (listening, seeing, manipulating, etc.)? 

2. Flow much information and/or training do you need about differentiating presentation of information for various 
ability levels (gifted, students with disabilities, etc.)? 

3. Flow much information and/or training do you need about differentiating presentation of information for varied lev 
els of English language proficiency? 

4. Flow much information and/or training do you need about adapting learning activities to engage students of varied 
learning styles (listening, seeing, manipulating, etc.)? 

5. Flow much information and/or training do you need about adapting learning activities to engage students of various 
ability levels (gifted, students with disabilities, etc.)? 

6. Flow much information and/or training do you need about adapting learning activities to engage students of varied 
levels of English language proficiency? 

7. Flow much information and/or training do you need about allowing students to demonstrate learning in ways that 
accommodate varied learning styles (seeing, listening, manipulating, etc.)? 

8. Flow much information and/or training do you need about allowing students to demonstrate learning in ways that 
accommodate varied ability levels (gifted, students with disabilities, etc.)? 

9. Flow much information and/or training do you need about allowing students to demonstrate learning in ways that 
accommodate varied levels of English language proficiency? 

10. Flow much information and/or training do you need to find research-based articles and/or books on practices rel¬ 
evant to specific educational needs of students? 

11. Flow much information and/or training do you need to judge the trustworthiness of research-based articles or books 
about effectiveness of educational practices? 
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12. How much information and/or training do you need to evaluate whether the research-based practices are 
worthwhile for my specific students and purposes? 

13. How much information and/or training do you need to compare effectiveness of research-based educational 
practices for the best fit for my particular student population? 

14. How much information and/or training do you need about changing educational practice to incorporate new 
instructional practices found in a research-based article or book? 

15. How much information and/or training do you need to work with a team(s) of grade-level or content-specific 
educators to assess specific learning needs? 

16. How much information and/or training do you need to work with a team(s) of grade-level or content-specific 
educators to solve specific learning needs? 

17. How much information and/or training do you need to collaborate with professionals outside my own field of 
specialty to assess specific learning needs (for example, teachers working with school psychologists or guidance 
counselors)? 

18. How much information and/or training do you need to collaborate with professionals outside my own field of 
specialty to solve specific learning needs (for example, teachers working with school psychologists or guidance 
counselors)? 

19. How much information and/or training do you need to use data from appropriate assessment tools to clarify the 
specific problem for a struggling student? 

20. How much information and/or training do you need to use specific assessments to measure student progress on 
specific learning objectives? 

21. How much information and/or training do you need to use results of universal screening instruments (like PALS, 

DIAL-R, or DIBELS) to determine which students may be at risk of specific learning needs? 

22. How much information and/or training do you need to use results of published curriculum-based assessments for 
instructional planning (like textbook assessments, PALS quick checks, etc.)? 

23. How much information and/or training do you need to make decisions about academic instruction for individual 
students based upon data? 

24. How much information and/or training do you need to make decisions about behavioral instruction for individual 
students based upon data? 

25. How much information and/or training do you need to use data on student progress to improve instructional 
practice? 

26. How much information and/or training do you need to use teaching techniques described in a research-based article 
or book? 

27. How much information and/or training do you need to use interventions to address specific learning objectives of 
specific students? 

28. How much information and/or training do you need to implement plans as designed to solve problems for individual 
students or small groups of students? 

29. How much information and/or training do you need to respond to a learning need when first evident? 
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Appendix B 

Scale Quality Indicators 


Table A1 

Total Variance Explained 


Initial Eigenvalues from Principal Component Analysis 



%of 

Cumulative 



%of 

Cumulative 

Component 

Total 

Variance 

% 

Component 

Total 

Variance 

% 

1 

24.947 

43.013 

43.013 

30 

.164 

.283 

97.978 

2 

4.808 

8.289 

51.302 

31 

.150 

.259 

98.238 

3 

4.187 

7.220 

58.521 

32 

.123 

.212 

98.450 

4 

3.581 

6.174 

64.696 

33 

.103 

.178 

98.628 

5 

2.420 

4.173 

68.869 

34 

TOO 

.173 

98.801 

6 

2.226 

3.838 

72.707 

35 

.081 

.139 

98.940 

7 

1.531 

2.640 

75.347 

36 

.076 

.130 

99.071 

8 

1.295 

2.233 

77.580 

37 

.065 

.112 

99.183 

9 

1.192 

2.055 

79.635 

38 

.054 

.094 

99.276 

10 

1.070 

1.845 

81.480 

39 

.051 

.088 

99.365 

11 

.991 

1.709 

83.189 

40 

.045 

.077 

99.441 

12 

.821 

1.416 

84.605 

41 

.040 

.069 

99.511 

13 

.793 

1.368 

85.973 

42 

.036 

.062 

99.573 

14 

.725 

1.251 

87.223 

43 

.033 

.058 

99.631 

15 

.709 

1.223 

88.446 

44 

.027 

.047 

99.678 

16 

.686 

1.183 

89.629 

45 

.026 

.044 

99.722 

17 

.570 

.982 

90.611 

46 

.024 

.042 

99.764 

18 

.537 

.926 

91.537 

47 

.022 

.038 

99.802 

19 

.525 

.904 

92.442 

48 

.021 

.036 

99.838 

20 

.455 

.785 

93.226 

49 

.016 

.028 

99.866 

21 

.409 

.704 

93.931 

50 

.016 

.027 

99.893 

22 

.387 

.667 

94.598 

51 

.013 

.023 

99.916 

23 

.356 

.614 

95.211 

52 

.011 

.019 

99.935 

24 

.319 

.549 

95.761 

53 

.009 

.015 

99.950 

25 

.297 

.513 

96.273 

54 

.008 

.014 

99.964 

26 

.257 

.443 

96.717 

55 

.007 

.013 

99.977 

27 

.201 

.347 

97.064 

56 

.006 

.011 

99.988 

28 

.188 

.324 

97.388 

57 

.005 

.008 

99.996 

29 

.178 

.307 

97.695 

58 

.002 

.004 

100.000 
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Appendix C 

Pearsonian Item-Total Correlations for RTISES 


, Correlation with Total Score 

Number 



Information Ouestion 

Training Ouestion 

1 

.318** 

.359** 

2 

.322** 

.311** 

3 

.272** 

.252** 

4 

.401** 

.407** 

5 

.519** 

.538** 

6 

.365** 

.356** 

7 

.438** 

.464** 

8 

.505** 

.516** 

9 

.377** 

.386** 

10 

.515** 

.519** 

11 

.538** 

.580** 

12 

.616** 

.598** 

13 

.679** 

.649** 

15 

.686** 

.669** 

16 

.642** 

.666** 

17 

.654** 

.654** 

18 

.655** 

.676** 

19 

.633** 

.663** 

20 

.702** 

.705** 

21 

.671** 

.675** 

22 

.577** 

.593** 

23 

.607** 

.600** 

24 

.667** 

.655** 

25 

.570** 

.603** 

26 

.707** 

.720** 

27 

.665** 

.672** 

28 

.712** 

.699** 

29 

.692** 

.696** 

30 

.682** 

.707** 


** Correlation is significant at the 0.001 level (2-tailed). 


Note: Missing data deleted pairwise. Number of respondents range is 155 to 174. 
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Appendix D 

Rotated Component Matrix for Initial Scale 


Component 


Item 

1 

2 

3 

4 

5 

1 information 

.158 

.761 

.105 

.003 

.244 

1 training 

.208 

.691 

.183 

.111 

.222 

2 information 

.150 

.762 

.144 

.193 

.007 

2 training 

.206 

.708 

.157 

.211 

-.008 

3 information 

.051 

.267 

.122 

.794 

.045 

3 training 

.049 

.162 

.139 

.824 

.014 

4 information 

.200 

.758 

.196 

.017 

.161 

4 training 

.212 

.738 

.238 

.098 

.071 

5 information 

.287 

.671 

.245 

.240 

.098 

5 training 

.293 

.646 

.283 

.270 

.110 

6 information 

.105 

.177 

.172 

.861 

.149 

6 training 

.108 

.146 

.182 

.886 

.116 

7 information 

.137 

.782 

.129 

.171 

.121 

7 training 

.128 

.739 

.201 

.194 

.124 

8 information 

.207 

.791 

.109 

.267 

.066 

8 training 

.200 

.772 

.209 

.255 

.126 

9 information 

.095 

.181 

.110 

.878 

.128 

9 training 

.099 

.167 

.145 

.887 

.122 

10 information 

.201 

.236 

.761 

.019 

.139 

10 training 

.190 

.230 

.765 

.019 

.162 

11 information 

.111 

.195 

.864 

.112 

.111 

11 training 

.155 

.207 

.848 

.149 

.138 

12 information 

.224 

.134 

.851 

.137 

.122 

12 training 

.226 

.131 

.856 

.135 

.142 

13 information 

.233 

.169 

.788 

.239 

.084 

13 training 

.246 

.147 

.776 

.250 

.097 

14 information 

.213 

.345 

.727 

.158 

.151 

14 training 

.221 

.343 

.696 

.157 

.132 

15 information 

.460 

.453 

.213 

-.087 

.355 

15 training 

.497 

.426 

.273 

-.107 

.353 

16 information 

.474 

.378 

.192 

.006 

.445 

17 training 

.494 

.314 

.237 

.001 

.460 

17 information 

.228 

.227 

.238 

.142 

.835 

18 training 

.233 

.217 

.275 

.180 

.820 

18 information 

.201 

.192 

.201 

.215 

.838 

19 training 

.244 

.144 

.236 

.299 

.795 

19 information 

.560 

.484 

.107 

.062 

.226 

20 training 

.582 

.458 

.101 

.062 

.252 

20 information 

.633 

.308 

.177 

-.006 

.244 

20 training 

.657 

.273 

.202 

-.008 

.243 

21 information 

.714 

.183 

.085 

-.064 

.114 

21 training 

.726 

.178 

.089 

-.058 

.144 

22 information 

.824 

-.037 

.202 

.087 

.086 

22 training 

.825 

-.041 

.187 

.093 

.082 

23 information 

.826 

.181 

.192 

.139 

-.040 

23 training 

.828 

.152 

.208 

.123 

-.043 

24 information 

.480 

.125 

.186 

.375 

.117 

24 training 

.485 

.098 

.270 

.369 

.147 

25 information 

.719 

.394 

.143 

.043 

.057 

25 training 

.736 

.389 

.129 

.096 

.041 

26 information 

.509 

.065 

.458 

.204 

.220 

26 training 

.490 

.075 

.486 

.201 

.223 

27 information 

.527 

.409 

.179 

.301 

.096 

27 training 

.524 

.364 

.183 

.324 

.104 

28 information 

.451 

.349 

.199 

.347 

.204 

28 training 

.482 

.310 

.190 

.385 

.194 

29 information 

.566 

.243 

.187 

.257 

.238 

29 training 

.609 

.218 

.216 

.284 

.205 


Extraction Method: Principal Component Analysis. Rotation Method: Varimax with Kaiser 
Normalization, a. Rotation converged in 7 iterations. 

Note: Each item appears twice - addressing the need for information and for training 
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Appendix E 

Correlations between Items Addressing the Need for Information and the 
Need for Training in the Same Professional Development Content Area 

Item Number and Professional Development Content Area Corr. 

1. differentiating presentation of information for various learning styles .851 

2. differentiating presentation of information for various ability levels .872 

3. differentiating presentation for varied levels of English language proficiency .905 

4. adapting learning activities to engage students of varied learning styles .899 

5. adapting learning activities to engage students of various ability levels .933 

6. adapting activities to engage ELL students .961 

7. allowing students to demonstrate learning in ways that accommodate learning styles .869 

8. allowing students to demonstrate learning in ways that accommodate ability levels .906 

9. allowing students to demonstrate learning in ways that accommodate ELL .948 

10. finding research-based articles and/or books on practices .951 

11. judging the trustworthiness of research-based articles or books .950 

12. evaluating whether the research-based practices are worthwhile .959 

13. comparing effectiveness of research-based educational practices for the best fit .918 

14. changing practice to incorporate new practices found in a research-based article .945 

15. working with a team(s) of grade-level or content-specific educators to assess needs .908 

16. working with a team(s) of grade-level or content-specific educators to solve needs .911 

17. collaborating with professionals outside my field to assess learning needs .951 

18. collaborating with professionals outside my field to solve specific learning needs .948 

19. using data from appropriate assessment tools to clarify the specific problem .916 

20. using specific assessments to measure student progress .906 

21. using results of universal screening instruments .988 

22. using results of published curriculum-based assessments for instructional planning .994 

23. making decisions about academic instruction for individual students based upon data .963 

24. making decisions about behavioral instruction for students based upon data .888 

25. using data on student progress to improve instructional practice .960 

26. using teaching techniques described in a research-based article or book .950 

27. using interventions to address specific learning objectives of specific students .927 

28. implementing plans as designed to solve problems for students .942 

29. responding to a learning need when first evident .930 

Note: All correlations are significant at the 0.01 level (2-tailed). 
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Appendix F 

Rotated Component Matrix for Revised Scale 



LonxononL 

1 

2 

3 

4 

3 

6 


.133 

.722 

.161 

.030 

.294 

220 

L’DrepAbtraiti 

.113 

.742 

.120 

.116 

.300 

-.021 

IJA.’U'.MW.J' 

.193 

.753 

.242 

049 

.15* 

.053 

lT3e_tw^Ab<rai_n 

.295 

.6*3 

2*2 

263 

.144 

.0*3 

URaBaLShw 

. 152 

.774 

.182 

.198 

.014 

.166 


217 

.771 

.191 

256 

.037 

.171 

CEtepELLtnun 

.019 

.193 

.135 

.830 

.124 

.010 

IJA.'uUi Jur. 

.078 

.158 

.171 

ah: 

.159 

.146 

IBwBIMlH 

.089 

.181 

.146 

.711 

.085 

.148 

EBiitnXDlraio 

.185 

.251 

.771 

.013 

.037 

.16* 

miitTRUSTtrain 

.162 

219 

.*48 

.170 

.064 

.127 


.209 

.127 

.865 

.134 

.145 

.140 

EBHtCOMPARE train 

.200 

.143 

.7*2 

232 

.190 

.101 

EBIiCHANGliriin 

.151 

140 

.694 

119 

.268 

.137 


.622 

.183 

.275 

-.067 

-.02* 

J37 

CtilltibOraJeSolvetraia 

.316 

260 

.229 

010 

.120 

.480 


.22* 

202 

.270 

.148 

.177 

4U6 


.168 

.154 

.232 

236 

.285 

.783 

Entail iistuMtictraic 

.699 

.420 

.too 

.109 

.047 

269 

JW.nx'rt'i'rtvrj 

.7*7 

250 

.201 

023 

.099 

.256 

f^mSprenewp 

.779 

.120 

.112 

.012 

.108 

.106 

LtiuiPljntrj*. 

.772 

-.077 

.208 

.112 

.307 

.023 

DataAcjiieioictraiii 

.721 

133 

.206 

.113 

.417 

-.0*7 


.698 

.367 

.121 

.100 

.282 

.009 

In lervec .inaiF idel i t' xj:n 

.247 

091 

.441 

033 

.646 

221 

luNmwwCtoiciii 1 

hWjmWShfeSM 

In lenen :kM)*rarl vtr ain 

.294 

.212 

.40) 

M9 

.129 

213 

.131 

.131 

.1*2 

.174 

218 

.172 

.6*5 

.7*2 

.615 

.113 

215 

.198 


Piar.utiuc Mrttxnl Principal Compcincnt Arulyut 

Rotation Met'!*.id Varinua with K-i.iw.-r Ncimui imion. 
Rotation tort'eraeti in 7 harjtiixv 
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Appendix G 

MTISES, Multi-Tiered Instruction Self-Efficacy Scale 
(Also known as the RTISES-II, Response to Intervention Self-Efficacy Scale-II) 

All scale items use the following response options: 

I'm starting to 

get it, but I I do this, but I 

intake want lots could benefit I dont feel the I feel ready to 
anything more from more need for more help others 

o o o o o 

DIRECTIONS: 

For most of the following questions, you will be asked to indicate your needs for professional development in various 

educational practices. Please indicate the level of professional development you feel you need for each item. 

1. How much professional development do you need about differentiating presentation of information for various learn 
ing styles (listening, seeing, manipulating, etc.)? 

2. How much professional development do you need about differentiating presentation of information for various abil 
ity levels (gifted, students with disabilities, etc.)? 

3. How much professional development do you need about differentiating presentation of information for varied levels 
of English language proficiency? 

4. How much professional development do you need about adapting learning activities to engage students of varied 
learning styles (listening, seeing, manipulating, etc.)? 

5. How much professional development do you need about adapting learning activities to engage students of various 
ability levels (gifted, students with disabilities, etc.)? 

6. How much professional development do you need about adapting learning activities to engage students of varied 
levels of English language proficiency? 

7. IIow much professional development do you need about allowing students to demonstrate learning in ways that ac 
commodate varied learning styles (seeing, listening, manipulating, etc.)? 

8. IIow much professional development do you need about allowing students to demonstrate learning in ways that ac 
commodate varied ability levels (gifted, students with disabilities, etc.)? 

9. IIow much professional development do you need about allowing students to demonstrate learning in ways that ac 
commodate varied levels of English language proficiency? 

10. IIow much professional development do you need to find research-based articles and/or books on practices relevant 
to specific educational needs of students? 

11. IIow much professional development do you need to judge the trustworthiness of research-based articles or books 
about effectiveness of educational practices? 

12. IIow much professional development do you need to evaluate whether the research-based practices are worth 
while for my specific students and purposes? 

13. IIow much professional development do you need to compare effectiveness of research-based educational practices 
for the best fit for my particular student population? 
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14. How much professional development do you need about changing educational practice to incorporate new instruc 
tional practices found in a research-based article or book? 

15. How much professional development do you need to work with a team(s) of grade-level or content-specific educators 
to assess specific learning needs? 

16. How much professional development do you need to work with a team(s) of grade-level or content-specific educators 
to solve specific learning needs? 

17. How much professional development do you need to collaborate with professionals outside my own field of specialty 
to assess specific learning needs (for example, teachers working with school psychologists or guidance counselors)? 

18. How much professional development do you need to collaborate with professionals outside my own field of specialty 
to solve specific learning needs (for example, teachers working with school psychologists or guidance counselors)? 

19. How much professional development do you need to use data from appropriate assessment tools to clarify the spe 
cific problem for a struggling student? 

20. How much professional development do you need to use specific assessments to measure student progress on spe 
cific learning objectives? 

21. How much professional development do you need to use results of universal screening instruments (like PALS, 
DIAL-R, or DIBELS) to determine which students may be at risk of specific learning needs? 

22. How much professional development do you need to use results of published curriculum-based assessments for in 
structional planning (like textbook assessments, PALS quick checks, etc.)? 

23. How much professional development do you need to make decisions about academic instruction for individual stu 
dents based upon data? 

24. How much professional development do you need to use data on student progress to improve instructional practice? 

25. How much professional development do you need to use teaching techniques described in a research-based article 
or book? 

26. How much professional development do you need to use interventions to address specific learning objectives of 
specific students? 

27. How much professional development do you need to implement plans as designed to solve problems for individual 
students or small groups of students? 

28. Plow much professional development do you need to respond to a learning need when first evident? 
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